Optimally Quantized and Smoothed Histograms

We propose an approach using optimal quantization and smoothing to generate adaptive histograms for multi-class one dimensional data. The discretization of data is optimal in maximizing a quantizer performance measure, defined by a combination of average log likelihood, entropy and correct classification probability. The optimal partition is found by dynamic programming. The density of each bin is obtained by a smoothing technique that can be considered a generalized k nearest neighbor density estimation algorithm. However, our smoothing approach is much more efficient. Experimental results demonstrated the effectiveness of the optimally quantized and smoothed histograms. Even though obtaining one takes about quadratic time in sample size, an optimal histogram is much more efficient to use than typical kernel methods. Therefore, optimal histograms are more suitable in applications with massive data set.